Data-local Reduce Task Scheduling
نویسندگان
چکیده
منابع مشابه
An Optimized Algorithm for Reduce Task Scheduling
In this paper, we propose a novel algorithm to solve the starving problem of the small jobs and reduce the process time of the small jobs on Hadoop platform. Current schedulers of MapReduce/Hadoop are quite successful in achieving data locality and scheduling the reduce tasks with a greedy algorithm. Some jobs may have hundreds of map tasks and just several reduce tasks, in which case, the redu...
متن کاملMaximizing Data Locality in Hadoop Clusters via Controlled Reduce Task Scheduling
The overall goal of this project is to gain a hands-on experience with working on a large open-ended research-oriented project using the Hadoop framework. Hadoop is an open source implementation of MapReduce and Google File System, and is currently enjoying wide popularity. Students will modify the task scheduler of Hadoop, conduct several experimental studies, and analyze performance and netwo...
متن کاملTask Scheduling in Data Stream Processing
One possible technique of data processing is its transformation into a data stream and the execution of particular operations on the data tuples. These operations can be usually processed concurrently especially when the plan of operations is branched. Therefore, this way of data processing is suitable for evaluation in parallel environment. On the other hand, the ordering of the execution of t...
متن کاملIncorporating Data Movement into Grid Task Scheduling
Task Scheduling is a critical design issue of distributed computing. The emerging Grid computing infrastructure consists of heterogeneous resources in widely distributed autonomous domains and makes task scheduling even more challenging. Grid considers both static, unmovable hardware and moveable, replicable data as computing resources. While intensive research has been done on task scheduling ...
متن کاملEnergy Aware Task Scheduling in Data Centers
Nowadays energy consumption problem is a major issue for data centers. The energy consumption increases significantly along with its CPU frequency getting higher. With Dynamic Voltage and Frequency Scaling (DVFS) techniques, CPU could be set to a suitable working frequency during the running time according to the workload. On the other side, reducing frequency implies that more servers will be ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Procedia Computer Science
سال: 2016
ISSN: 1877-0509
DOI: 10.1016/j.procs.2016.05.226